@MichaelIngvarRoenning Sorry for the delay, I've been heads down on updating some other training stuff & redesign for our homepage. To keep you updated while I make some changes:

1a. in notebook 111 you are talking about Key Concepts: Model Documentation is not relevant for Validation. it's Validation Report Documentation that is what we are focusing on here. Same with documentation template concept, we are not using test suits from a template in these notebooks.

Agreed, I will give these introductory concepts another edit to make them more tailored to model validation.

EDIT: Done!

1b. In general the language needs to be changed I see instances of cases where we say "was used to develop our dummy champion model" - we need the language to be that Model Developer has submitted their model to the validation team, and that champion model from developer is subject for testing from the Validation Team.

Will do, I will massage the messaging around this.

EDIT: Done!

1c. When we are pre-processing and doing feature engineering of the dataset - there needs to be emphasis in the description that Model Validator is not replicating what the Model Developer did for dataset adjustments, and we want to replicate this as we want to test if the transformations and processing of data was appropriate.

Got it, we are reproducing the steps the model developer WOULD have taken to see if they were done correctly, not just mimicking, correct?

EDIT: Done!

Made sure any sections referencing why we do this are clear etc.:

2a. Notebook 112 - I'm not sure why we are again importing a champion model when the section is develop challenger models. The Notebook should be clear that we are ALSO registering the champion model in the framework.
3a. Isn't 111 and 112 redundant since we are doing exactly the same in 113? what's the point of doing 111, 112 and then 113?

Same comment here as mentioned in Slack — unfortunately each notebook are "isolated" environments, so certain functions/inputs/outputs need to be rerun in order to be accessed. If you comment out those sections it won't work — very frustrating, I know, but I ran into this in the last series. It's just quirk of Jupyter Notebooks unfortunately.

2b. I think it makes more sense to merge notebook 111 and 112 into one piece to be honest, that makes more sense in this setup.
I recommend that we start with 113 with basis, and include what is needed from 111 and 112 and then end it with 114.

I would like to keep it as is — the reason is we are structuring our updated training around this breakdown and it also matches the development introductory experience. e.g.

I think that once I clean up 111 to be more validator focused it will make more sense like this though!

2c. I think the last part of Enable use-case context should come when we actually do a test where it becomes relevant. These notebooks should not be an "intro for validator" this should be more related to validation of credit risk application where we are assuming they knowledgable users.

• I thought it was out of place too, can you recommend where you would actually introduce this? Should we just remove the first instance and leave the second one where we modify description for the comparison tests?
• We are actually trying to develop an introduction for validation experience first (to work with the training), so I think we should focus on the basics for this initiative and then follow-up with a specific more complex use case notebook. With that in mind, how do you think we can approach this differently in your recommendation?

3b. When running the list of tests, I actually recommend we run ONE test first to see and then we run the loop of all tests. this is relevant for every loop we have here. The rest is great

Can you provide an example here of how you might structure it? I broke them down as best I could but I don't know enough about which tests are valuable to make a call.

PR Summary

This pull request introduces a new Jupyter notebook for validating an application scorecard model using the ValidMind Library. The notebook provides a comprehensive guide for assessing the model's development through testing and analysis, including the use of challenger models for benchmarking performance. Key features of the notebook include:

• Instructions for setting up the environment and initializing the ValidMind Library.
• Steps for importing and preprocessing a sample dataset.
• Training of potential challenger models (Random Forest and Logistic Regression) for comparison.
• Running various data quality and performance tests using the ValidMind Library.
• Implementing a custom test for score-to-odds analysis.
• Logging and verifying test results within the ValidMind Platform.

Additionally, the PR updates multiple documentation references across various notebooks to include information about running tests and accessing the Python Library API reference. It also updates the .gitignore file to ensure a specific model file is not ignored.

The version of the ValidMind Library is incremented from 2.8.13 to 2.8.14.

Test Suggestions

• Run the new model validation notebook to ensure all steps execute without errors.
• Verify that the custom test for score-to-odds analysis produces the expected output.
• Check that the updated documentation references are correct and lead to the intended resources.
• Ensure that the .gitignore changes correctly include the specified model file.